[NFC] Replace more DenseMaps with SmallDenseMaps #111836
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
These DenseMaps all appear as some of the most frequent sources of memory-allocations that could otherwise be accomodated with an initial stack allocation. For simplicity, I've typedef'd one map-type to be BlockColorMapT, used by various block colouring functions. This also features one opportunistic replacement of std::vector with SmallVector, as it's in a path that obviously always allocates.
llvmbot
added
clang
Member
|
@llvm/pr-subscribers-pgo @llvm/pr-subscribers-backend-x86 Author: Jeremy Morse (jmorse) ChangesThese DenseMaps all appear as some of the most frequent sources of memory-allocations that could otherwise be accomodated with an initial stack allocation. For simplicity, I've typedef'd one map-type to be BlockColorMapT, used by various block colouring functions. This also features one opportunistic replacement of std::vector with SmallVector, as it's in a path that obviously always allocates. Compared to the other patches reducing DenseMap allocations the benefits from this one are quite small -- I've reached the point of diminishing returns: https://llvm-compile-time-tracker.com/compare.php?from=c7fbcae72557cbe14bca615038254649c1177401&to=0785a8d0fd31541d6a8af616111312780e268611&stat=instructions:u Patch is 27.85 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/111836.diff 32 Files Affected:
diff --git a/clang/include/clang/AST/CXXInheritance.h b/clang/include/clang/AST/CXXInheritance.h
index bbef01843e0b0a..b9057e15a41988 100644
--- a/clang/include/clang/AST/CXXInheritance.h
+++ b/clang/include/clang/AST/CXXInheritance.h
@@ -268,7 +268,7 @@ struct UniqueVirtualMethod {
/// subobject in which that virtual function occurs).
class OverridingMethods {
using ValuesT = SmallVector<UniqueVirtualMethod, 4>;
- using MapType = llvm::MapVector<unsigned, ValuesT>;
+ using MapType = llvm::SmallMapVector<unsigned, ValuesT, 16>;
MapType Overrides;
diff --git a/clang/lib/AST/ItaniumMangle.cpp b/clang/lib/AST/ItaniumMangle.cpp
index 777cdca1a0c0d7..0ec0855cf52440 100644
--- a/clang/lib/AST/ItaniumMangle.cpp
+++ b/clang/lib/AST/ItaniumMangle.cpp
@@ -392,8 +392,8 @@ class CXXNameMangler {
AbiTagState *AbiTags = nullptr;
AbiTagState AbiTagsRoot;
- llvm::DenseMap<uintptr_t, unsigned> Substitutions;
- llvm::DenseMap<StringRef, unsigned> ModuleSubstitutions;
+ llvm::SmallDenseMap<uintptr_t, unsigned, 16> Substitutions;
+ llvm::SmallDenseMap<StringRef, unsigned, 16> ModuleSubstitutions;
ASTContext &getASTContext() const { return Context.getASTContext(); }
diff --git a/llvm/include/llvm/ADT/SCCIterator.h b/llvm/include/llvm/ADT/SCCIterator.h
index 3bd103c13f19f9..a35e0a3f8e8c95 100644
--- a/llvm/include/llvm/ADT/SCCIterator.h
+++ b/llvm/include/llvm/ADT/SCCIterator.h
@@ -73,7 +73,7 @@ class scc_iterator : public iterator_facade_base<
///
/// nodeVisitNumbers are per-node visit numbers, also used as DFS flags.
unsigned visitNum;
- DenseMap<NodeRef, unsigned> nodeVisitNumbers;
+ SmallDenseMap<NodeRef, unsigned, 16> nodeVisitNumbers;
/// Stack holding nodes of the SCC.
std::vector<NodeRef> SCCNodeStack;
diff --git a/llvm/include/llvm/Analysis/ConstraintSystem.h b/llvm/include/llvm/Analysis/ConstraintSystem.h
index 449852343964ca..2ac0fc1ddc06c3 100644
--- a/llvm/include/llvm/Analysis/ConstraintSystem.h
+++ b/llvm/include/llvm/Analysis/ConstraintSystem.h
@@ -51,7 +51,7 @@ class ConstraintSystem {
/// A map of variables (IR values) to their corresponding index in the
/// constraint system.
- DenseMap<Value *, unsigned> Value2Index;
+ SmallDenseMap<Value *, unsigned, 16> Value2Index;
// Eliminate constraints from the system using Fourier–Motzkin elimination.
bool eliminateUsingFM();
@@ -70,7 +70,7 @@ class ConstraintSystem {
Value2Index.insert({Arg, Value2Index.size() + 1});
}
}
- ConstraintSystem(const DenseMap<Value *, unsigned> &Value2Index)
+ ConstraintSystem(const SmallDenseMap<Value *, unsigned, 16> &Value2Index)
: NumVariables(Value2Index.size()), Value2Index(Value2Index) {}
bool addVariableRow(ArrayRef<int64_t> R) {
@@ -92,8 +92,8 @@ class ConstraintSystem {
return true;
}
- DenseMap<Value *, unsigned> &getValue2Index() { return Value2Index; }
- const DenseMap<Value *, unsigned> &getValue2Index() const {
+ SmallDenseMap<Value *, unsigned, 16> &getValue2Index() { return Value2Index; }
+ const SmallDenseMap<Value *, unsigned, 16> &getValue2Index() const {
return Value2Index;
}
diff --git a/llvm/include/llvm/Analysis/DominanceFrontier.h b/llvm/include/llvm/Analysis/DominanceFrontier.h
index 68ddcf753b59f7..394379d9750827 100644
--- a/llvm/include/llvm/Analysis/DominanceFrontier.h
+++ b/llvm/include/llvm/Analysis/DominanceFrontier.h
@@ -41,8 +41,8 @@ class DominanceFrontierBase {
public:
// Dom set for a bb. Use SetVector to make iterating dom frontiers of a bb
// deterministic.
- using DomSetType = SetVector<BlockT *>;
- using DomSetMapType = DenseMap<BlockT *, DomSetType>; // Dom set map
+ using DomSetType = SmallSetVector<BlockT *, 16>;
+ using DomSetMapType = SmallDenseMap<BlockT *, DomSetType, 16>; // Dom set map
protected:
using BlockTraits = GraphTraits<BlockT *>;
diff --git a/llvm/include/llvm/Analysis/DominanceFrontierImpl.h b/llvm/include/llvm/Analysis/DominanceFrontierImpl.h
index e877b2c4749abe..165e137a5ecfc0 100644
--- a/llvm/include/llvm/Analysis/DominanceFrontierImpl.h
+++ b/llvm/include/llvm/Analysis/DominanceFrontierImpl.h
@@ -55,7 +55,7 @@ void DominanceFrontierBase<BlockT, IsPostDom>::print(raw_ostream &OS) const {
OS << " <<exit node>>";
OS << " is:\t";
- const SetVector<BlockT *> &BBs = I->second;
+ const SmallSetVector<BlockT *, 16> &BBs = I->second;
for (const BlockT *BB : BBs) {
OS << ' ';
diff --git a/llvm/include/llvm/Analysis/LoopIterator.h b/llvm/include/llvm/Analysis/LoopIterator.h
index 523d2a21825d0d..aaca0298d21da4 100644
--- a/llvm/include/llvm/Analysis/LoopIterator.h
+++ b/llvm/include/llvm/Analysis/LoopIterator.h
@@ -97,8 +97,8 @@ struct LoopBodyTraits {
class LoopBlocksDFS {
public:
/// Postorder list iterators.
- typedef std::vector<BasicBlock*>::const_iterator POIterator;
- typedef std::vector<BasicBlock*>::const_reverse_iterator RPOIterator;
+ typedef SmallVector<BasicBlock*, 16>::const_iterator POIterator;
+ typedef SmallVector<BasicBlock*, 16>::const_reverse_iterator RPOIterator;
friend class LoopBlocksTraversal;
@@ -108,8 +108,8 @@ class LoopBlocksDFS {
/// Map each block to its postorder number. A block is only mapped after it is
/// preorder visited by DFS. It's postorder number is initially zero and set
/// to nonzero after it is finished by postorder traversal.
- DenseMap<BasicBlock*, unsigned> PostNumbers;
- std::vector<BasicBlock*> PostBlocks;
+ SmallDenseMap<BasicBlock*, unsigned, 16> PostNumbers;
+ SmallVector<BasicBlock*, 16> PostBlocks;
public:
LoopBlocksDFS(Loop *Container) :
diff --git a/llvm/include/llvm/Analysis/MemorySSA.h b/llvm/include/llvm/Analysis/MemorySSA.h
index 09fc34af60dc3c..1ceabfa1b13cf9 100644
--- a/llvm/include/llvm/Analysis/MemorySSA.h
+++ b/llvm/include/llvm/Analysis/MemorySSA.h
@@ -879,7 +879,7 @@ class MemorySSA {
Loop *L = nullptr;
// Memory SSA mappings
- DenseMap<const Value *, MemoryAccess *> ValueToMemoryAccess;
+ SmallDenseMap<const Value *, MemoryAccess *, 16> ValueToMemoryAccess;
// These two mappings contain the main block to access/def mappings for
// MemorySSA. The list contained in PerBlockAccesses really owns all the
@@ -895,7 +895,7 @@ class MemorySSA {
// Note that the numbering is local to a block, even though the map is
// global.
mutable SmallPtrSet<const BasicBlock *, 16> BlockNumberingValid;
- mutable DenseMap<const MemoryAccess *, unsigned long> BlockNumbering;
+ mutable SmallDenseMap<const MemoryAccess *, unsigned long, 16> BlockNumbering;
// Memory SSA building info
std::unique_ptr<ClobberWalkerBase> WalkerBase;
diff --git a/llvm/include/llvm/Analysis/MustExecute.h b/llvm/include/llvm/Analysis/MustExecute.h
index 8ac3c5eb653cd4..e49b6facf76b53 100644
--- a/llvm/include/llvm/Analysis/MustExecute.h
+++ b/llvm/include/llvm/Analysis/MustExecute.h
@@ -58,7 +58,7 @@ class raw_ostream;
/// methods except for computeLoopSafetyInfo is undefined.
class LoopSafetyInfo {
// Used to update funclet bundle operands.
- DenseMap<BasicBlock *, ColorVector> BlockColors;
+ BlockColorMapT BlockColors;
protected:
/// Computes block colors.
@@ -66,7 +66,7 @@ class LoopSafetyInfo {
public:
/// Returns block colors map that is used to update funclet operand bundles.
- const DenseMap<BasicBlock *, ColorVector> &getBlockColors() const;
+ const BlockColorMapT &getBlockColors() const;
/// Copy colors of block \p Old into the block \p New.
void copyColors(BasicBlock *New, BasicBlock *Old);
diff --git a/llvm/include/llvm/IR/EHPersonalities.h b/llvm/include/llvm/IR/EHPersonalities.h
index c70f832de40b40..2dcb7511ba4dfc 100644
--- a/llvm/include/llvm/IR/EHPersonalities.h
+++ b/llvm/include/llvm/IR/EHPersonalities.h
@@ -107,12 +107,13 @@ inline bool isNoOpWithoutInvoke(EHPersonality Pers) {
bool canSimplifyInvokeNoUnwind(const Function *F);
typedef TinyPtrVector<BasicBlock *> ColorVector;
+typedef SmallDenseMap<BasicBlock *, ColorVector, 16> BlockColorMapT;
/// If an EH funclet personality is in use (see isFuncletEHPersonality),
/// this will recompute which blocks are in which funclet. It is possible that
/// some blocks are in multiple funclets. Consider this analysis to be
/// expensive.
-DenseMap<BasicBlock *, ColorVector> colorEHFunclets(Function &F);
+BlockColorMapT colorEHFunclets(Function &F);
} // end namespace llvm
diff --git a/llvm/include/llvm/IR/PredIteratorCache.h b/llvm/include/llvm/IR/PredIteratorCache.h
index ba3228347076b8..055ab91a522668 100644
--- a/llvm/include/llvm/IR/PredIteratorCache.h
+++ b/llvm/include/llvm/IR/PredIteratorCache.h
@@ -26,7 +26,7 @@ namespace llvm {
/// wants the predecessor list for the same blocks.
class PredIteratorCache {
/// Cached list of predecessors, allocated in Memory.
- DenseMap<BasicBlock *, ArrayRef<BasicBlock *>> BlockToPredsMap;
+ SmallDenseMap<BasicBlock *, ArrayRef<BasicBlock *>, 16> BlockToPredsMap;
/// Memory - This is the space that holds cached preds.
BumpPtrAllocator Memory;
diff --git a/llvm/include/llvm/Transforms/Utils/Cloning.h b/llvm/include/llvm/Transforms/Utils/Cloning.h
index a4be24e32c5279..3f27cc0cf449db 100644
--- a/llvm/include/llvm/Transforms/Utils/Cloning.h
+++ b/llvm/include/llvm/Transforms/Utils/Cloning.h
@@ -81,7 +81,7 @@ struct ClonedCodeInfo {
/// Like VMap, but maps only unsimplified instructions. Values in the map
/// may be dangling, it is only intended to be used via isSimplified(), to
/// check whether the main VMap mapping involves simplification or not.
- DenseMap<const Value *, const Value *> OrigVMap;
+ SmallDenseMap<const Value *, const Value *, 16> OrigVMap;
ClonedCodeInfo() = default;
diff --git a/llvm/include/llvm/Transforms/Utils/SSAUpdaterImpl.h b/llvm/include/llvm/Transforms/Utils/SSAUpdaterImpl.h
index 746926e5bee331..089a7d05ca0949 100644
--- a/llvm/include/llvm/Transforms/Utils/SSAUpdaterImpl.h
+++ b/llvm/include/llvm/Transforms/Utils/SSAUpdaterImpl.h
@@ -70,7 +70,7 @@ class SSAUpdaterImpl {
: BB(ThisBB), AvailableVal(V), DefBB(V ? this : nullptr) {}
};
- using AvailableValsTy = DenseMap<BlkT *, ValT>;
+ using AvailableValsTy = SmallDenseMap<BlkT *, ValT, 16>;
AvailableValsTy *AvailableVals;
diff --git a/llvm/lib/Analysis/InlineCost.cpp b/llvm/lib/Analysis/InlineCost.cpp
index d2c329ba748e58..3ad137382cd5fc 100644
--- a/llvm/lib/Analysis/InlineCost.cpp
+++ b/llvm/lib/Analysis/InlineCost.cpp
@@ -608,7 +608,7 @@ class InlineCostCallAnalyzer final : public CallAnalyzer {
/// The mapping of caller Alloca values to their accumulated cost savings. If
/// we have to disable SROA for one of the allocas, this tells us how much
/// cost must be added.
- DenseMap<AllocaInst *, int> SROAArgCosts;
+ SmallDenseMap<AllocaInst *, int, 16> SROAArgCosts;
/// Return true if \p Call is a cold callsite.
bool isColdCallSite(CallBase &Call, BlockFrequencyInfo *CallerBFI);
diff --git a/llvm/lib/Analysis/LazyValueInfo.cpp b/llvm/lib/Analysis/LazyValueInfo.cpp
index 30dc4ae30dbfa5..d7e663d60d8dc8 100644
--- a/llvm/lib/Analysis/LazyValueInfo.cpp
+++ b/llvm/lib/Analysis/LazyValueInfo.cpp
@@ -322,7 +322,7 @@ class LazyValueInfoImpl {
SmallVector<std::pair<BasicBlock*, Value*>, 8> BlockValueStack;
/// Keeps track of which block-value pairs are in BlockValueStack.
- DenseSet<std::pair<BasicBlock*, Value*> > BlockValueSet;
+ SmallDenseSet<std::pair<BasicBlock*, Value*>, 16> BlockValueSet;
/// Push BV onto BlockValueStack unless it's already in there.
/// Returns true on success.
diff --git a/llvm/lib/Analysis/MustExecute.cpp b/llvm/lib/Analysis/MustExecute.cpp
index caed62679a683c..52d631b56beb7b 100644
--- a/llvm/lib/Analysis/MustExecute.cpp
+++ b/llvm/lib/Analysis/MustExecute.cpp
@@ -28,7 +28,7 @@ using namespace llvm;
#define DEBUG_TYPE "must-execute"
-const DenseMap<BasicBlock *, ColorVector> &
+const BlockColorMapT &
LoopSafetyInfo::getBlockColors() const {
return BlockColors;
}
diff --git a/llvm/lib/CodeGen/LiveDebugValues/InstrRefBasedImpl.cpp b/llvm/lib/CodeGen/LiveDebugValues/InstrRefBasedImpl.cpp
index a9d28a39c4418b..2b65d2ed45b516 100644
--- a/llvm/lib/CodeGen/LiveDebugValues/InstrRefBasedImpl.cpp
+++ b/llvm/lib/CodeGen/LiveDebugValues/InstrRefBasedImpl.cpp
@@ -4151,7 +4151,7 @@ std::optional<ValueIDNum> InstrRefBasedLDV::resolveDbgPHIsImpl(
// Adapted LLVM SSA Updater:
LDVSSAUpdater Updater(Loc, MLiveIns);
// Map of which Def or PHI is the current value in each block.
- DenseMap<LDVSSABlock *, BlockValueNum> AvailableValues;
+ SmallDenseMap<LDVSSABlock *, BlockValueNum, 16> AvailableValues;
// Set of PHIs that we have created along the way.
SmallVector<LDVSSAPhi *, 8> CreatedPHIs;
diff --git a/llvm/lib/CodeGen/MachineSSAUpdater.cpp b/llvm/lib/CodeGen/MachineSSAUpdater.cpp
index 4cbb6ad3128bd9..186f729859092e 100644
--- a/llvm/lib/CodeGen/MachineSSAUpdater.cpp
+++ b/llvm/lib/CodeGen/MachineSSAUpdater.cpp
@@ -34,7 +34,7 @@ using namespace llvm;
#define DEBUG_TYPE "machine-ssaupdater"
-using AvailableValsTy = DenseMap<MachineBasicBlock *, Register>;
+using AvailableValsTy = SmallDenseMap<MachineBasicBlock *, Register, 16>;
static AvailableValsTy &getAvailableVals(void *AV) {
return *static_cast<AvailableValsTy*>(AV);
diff --git a/llvm/lib/CodeGen/WinEHPrepare.cpp b/llvm/lib/CodeGen/WinEHPrepare.cpp
index c58c67b70fe3c2..07057d4fe69375 100644
--- a/llvm/lib/CodeGen/WinEHPrepare.cpp
+++ b/llvm/lib/CodeGen/WinEHPrepare.cpp
@@ -90,7 +90,7 @@ class WinEHPrepareImpl {
EHPersonality Personality = EHPersonality::Unknown;
const DataLayout *DL = nullptr;
- DenseMap<BasicBlock *, ColorVector> BlockColors;
+ BlockColorMapT BlockColors;
MapVector<BasicBlock *, std::vector<BasicBlock *>> FuncletBlocks;
};
@@ -189,7 +189,7 @@ static BasicBlock *getCleanupRetUnwindDest(const CleanupPadInst *CleanupPad) {
static void calculateStateNumbersForInvokes(const Function *Fn,
WinEHFuncInfo &FuncInfo) {
auto *F = const_cast<Function *>(Fn);
- DenseMap<BasicBlock *, ColorVector> BlockColors = colorEHFunclets(*F);
+ BlockColorMapT BlockColors = colorEHFunclets(*F);
for (BasicBlock &BB : *F) {
auto *II = dyn_cast<InvokeInst>(BB.getTerminator());
if (!II)
diff --git a/llvm/lib/IR/EHPersonalities.cpp b/llvm/lib/IR/EHPersonalities.cpp
index 7c32601b8a83ea..08cbd92cc38baf 100644
--- a/llvm/lib/IR/EHPersonalities.cpp
+++ b/llvm/lib/IR/EHPersonalities.cpp
@@ -102,10 +102,10 @@ bool llvm::canSimplifyInvokeNoUnwind(const Function *F) {
return !EHa && !isAsynchronousEHPersonality(Personality);
}
-DenseMap<BasicBlock *, ColorVector> llvm::colorEHFunclets(Function &F) {
+BlockColorMapT llvm::colorEHFunclets(Function &F) {
SmallVector<std::pair<BasicBlock *, BasicBlock *>, 16> Worklist;
BasicBlock *EntryBlock = &F.getEntryBlock();
- DenseMap<BasicBlock *, ColorVector> BlockColors;
+ BlockColorMapT BlockColors;
// Build up the color map, which maps each block to its set of 'colors'.
// For any block B the "colors" of B are the set of funclets F (possibly
diff --git a/llvm/lib/IR/Verifier.cpp b/llvm/lib/IR/Verifier.cpp
index b89c9ce46e7d61..2d594945ca839f 100644
--- a/llvm/lib/IR/Verifier.cpp
+++ b/llvm/lib/IR/Verifier.cpp
@@ -363,7 +363,7 @@ class Verifier : public InstVisitor<Verifier>, VerifierSupport {
/// Cache which blocks are in which funclet, if an EH funclet personality is
/// in use. Otherwise empty.
- DenseMap<BasicBlock *, ColorVector> BlockEHFuncletColors;
+ BlockColorMapT BlockEHFuncletColors;
/// Cache of constants visited in search of ConstantExprs.
SmallPtrSet<const Constant *, 32> ConstantExprVisited;
diff --git a/llvm/lib/Target/X86/X86WinEHState.cpp b/llvm/lib/Target/X86/X86WinEHState.cpp
index 963d613ddbfe7d..199f8438c61e8e 100644
--- a/llvm/lib/Target/X86/X86WinEHState.cpp
+++ b/llvm/lib/Target/X86/X86WinEHState.cpp
@@ -70,9 +70,9 @@ class WinEHStatePass : public FunctionPass {
bool isStateStoreNeeded(EHPersonality Personality, CallBase &Call);
void rewriteSetJmpCall(IRBuilder<> &Builder, Function &F, CallBase &Call,
Value *State);
- int getBaseStateForBB(DenseMap<BasicBlock *, ColorVector> &BlockColors,
+ int getBaseStateForBB(BlockColorMapT &BlockColors,
WinEHFuncInfo &FuncInfo, BasicBlock *BB);
- int getStateForCall(DenseMap<BasicBlock *, ColorVector> &BlockColors,
+ int getStateForCall(BlockColorMapT &BlockColors,
WinEHFuncInfo &FuncInfo, CallBase &Call);
// Module-level type getters.
@@ -501,7 +501,7 @@ void WinEHStatePass::rewriteSetJmpCall(IRBuilder<> &Builder, Function &F,
// Figure out what state we should assign calls in this block.
int WinEHStatePass::getBaseStateForBB(
- DenseMap<BasicBlock *, ColorVector> &BlockColors, WinEHFuncInfo &FuncInfo,
+ BlockColorMapT &BlockColors, WinEHFuncInfo &FuncInfo,
BasicBlock *BB) {
int BaseState = ParentBaseState;
auto &BBColors = BlockColors[BB];
@@ -520,7 +520,7 @@ int WinEHStatePass::getBaseStateForBB(
// Calculate the state a call-site is in.
int WinEHStatePass::getStateForCall(
- DenseMap<BasicBlock *, ColorVector> &BlockColors, WinEHFuncInfo &FuncInfo,
+ BlockColorMapT &BlockColors, WinEHFuncInfo &FuncInfo,
CallBase &Call) {
if (auto *II = dyn_cast<InvokeInst>(&Call)) {
// Look up the state number of the EH pad this unwinds to.
@@ -644,7 +644,7 @@ void WinEHStatePass::addStateStores(Function &F, WinEHFuncInfo &FuncInfo) {
calculateWinCXXEHStateNumbers(&F, FuncInfo);
// Iterate all the instructions and emit state number stores.
- DenseMap<BasicBlock *, ColorVector> BlockColors = colorEHFunclets(F);
+ BlockColorMapT BlockColors = colorEHFunclets(F);
ReversePostOrderTraversal<Function *> RPOT(&F);
// InitialStates yields the state of the first call-site for a BasicBlock.
diff --git a/llvm/lib/Transforms/InstCombine/InstructionCombining.cpp b/llvm/lib/Transforms/InstCombine/InstructionCombining.cpp
index 1f4a6f793404cf..fbd11292115c2b 100644
--- a/llvm/lib/Transforms/InstCombine/InstructionCombining.cpp
+++ b/llvm/lib/Transforms/InstCombine/InstructionCombining.cpp
@@ -5307,7 +5307,7 @@ bool InstCombinerImpl::prepareWorklist(Function &F) {
bool MadeIRChange = false;
SmallPtrSet<BasicBlock *, 32> LiveBlocks;
SmallVector<Instruction *, 128> InstrsForInstructionWorklist;
- DenseMap<Constant *, Constant *> FoldedConstants;
+ SmallDenseMap<Constant *, Constant *, 16> FoldedConstants;
AliasScopeTracker SeenAliasScopes;
auto HandleOnlyLiveSuccessor = [&](BasicBlock *BB, BasicBlock *LiveSucc) {
diff --git a/llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp b/llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp
index 2ad89b5ba753a5..4665304632c9bd 100644
--- a/llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp
+++ b/llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp
@@ -669,7 +669,7 @@ class RuntimeCallInserter {
return;
assert(TrackInsertedCalls && "Calls were wrongly tracked");
- DenseMap<BasicBlock *, ColorVector> BlockColors = colorEHFunclets(*OwnerFn);
+ BlockColorMapT BlockColors = colorEHFunclets(*OwnerFn);
for (CallInst *CI : InsertedCalls) {
BasicBlock *BB = CI->getParent();
assert(BB && "Instruction doesn't belong to a BasicBlock");
diff --git a/llvm/lib/Transforms/Instrumentation/PGOInstrumentation.cpp b/llvm/lib/Transforms/Instrumentation/PGOInstrumentation.cpp
index 10442fa0bb9003..164ffbec26421a 100644
--- a/llvm/lib/Transforms/Instrumentation/PGOInstrumentation.cpp
+++ b/llvm/lib/Transforms/Instrumentation/PGOInstrumentation.cpp
@@ -864,7 +864,7 @@ BasicBlock *FuncPGOInstrumentation<Edge, BBInfo>::getInstrBB(Edge *E) {
// value profiling call for the value profile candidate call.
static void
populateEHOperandBundle(VPCandidateInfo &Cand,
- DenseMap<BasicBlock *, ColorVector> &BlockColors,
+ BlockColorMapT &BlockColors,
SmallVectorImpl<OperandBundleDef> &OpBundles) {
auto *OrigCall = dyn_cast<CallBase>(Cand.AnnotatedInst);
if (!OrigCall)
@@ -1006,7 +1006,7 @@ void FunctionInstrumenter::instrument() {
// Windows exception handling attached to them. However, if value profiling is
// inserted for one of these calls, then a funclet value will need to be set
// on the instrumenta...
[truncated]
|
Member
|
@llvm/pr-subscribers-debuginfo Author: Jeremy Morse (jmorse) ChangesThese DenseMaps all appear as some of the most frequent sources of memory-allocations that could otherwise be accomodated with an initial stack allocation. For simplicity, I've typedef'd one map-type to be BlockColorMapT, used by various block colouring functions. This also features one opportunistic replacement of std::vector with SmallVector, as it's in a path that obviously always allocates. Compared to the other patches reducing DenseMap allocations the benefits from this one are quite small -- I've reached the point of diminishing returns: https://llvm-compile-time-tracker.com/compare.php?from=c7fbcae72557cbe14bca615038254649c1177401&to=0785a8d0fd31541d6a8af616111312780e268611&stat=instructions:u Patch is 27.85 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/111836.diff 32 Files Affected:
diff --git a/clang/include/clang/AST/CXXInheritance.h b/clang/include/clang/AST/CXXInheritance.h
index bbef01843e0b0a..b9057e15a41988 100644
--- a/clang/include/clang/AST/CXXInheritance.h
+++ b/clang/include/clang/AST/CXXInheritance.h
@@ -268,7 +268,7 @@ struct UniqueVirtualMethod {
/// subobject in which that virtual function occurs).
class OverridingMethods {
using ValuesT = SmallVector<UniqueVirtualMethod, 4>;
- using MapType = llvm::MapVector<unsigned, ValuesT>;
+ using MapType = llvm::SmallMapVector<unsigned, ValuesT, 16>;
MapType Overrides;
diff --git a/clang/lib/AST/ItaniumMangle.cpp b/clang/lib/AST/ItaniumMangle.cpp
index 777cdca1a0c0d7..0ec0855cf52440 100644
--- a/clang/lib/AST/ItaniumMangle.cpp
+++ b/clang/lib/AST/ItaniumMangle.cpp
@@ -392,8 +392,8 @@ class CXXNameMangler {
AbiTagState *AbiTags = nullptr;
AbiTagState AbiTagsRoot;
- llvm::DenseMap<uintptr_t, unsigned> Substitutions;
- llvm::DenseMap<StringRef, unsigned> ModuleSubstitutions;
+ llvm::SmallDenseMap<uintptr_t, unsigned, 16> Substitutions;
+ llvm::SmallDenseMap<StringRef, unsigned, 16> ModuleSubstitutions;
ASTContext &getASTContext() const { return Context.getASTContext(); }
diff --git a/llvm/include/llvm/ADT/SCCIterator.h b/llvm/include/llvm/ADT/SCCIterator.h
index 3bd103c13f19f9..a35e0a3f8e8c95 100644
--- a/llvm/include/llvm/ADT/SCCIterator.h
+++ b/llvm/include/llvm/ADT/SCCIterator.h
@@ -73,7 +73,7 @@ class scc_iterator : public iterator_facade_base<
///
/// nodeVisitNumbers are per-node visit numbers, also used as DFS flags.
unsigned visitNum;
- DenseMap<NodeRef, unsigned> nodeVisitNumbers;
+ SmallDenseMap<NodeRef, unsigned, 16> nodeVisitNumbers;
/// Stack holding nodes of the SCC.
std::vector<NodeRef> SCCNodeStack;
diff --git a/llvm/include/llvm/Analysis/ConstraintSystem.h b/llvm/include/llvm/Analysis/ConstraintSystem.h
index 449852343964ca..2ac0fc1ddc06c3 100644
--- a/llvm/include/llvm/Analysis/ConstraintSystem.h
+++ b/llvm/include/llvm/Analysis/ConstraintSystem.h
@@ -51,7 +51,7 @@ class ConstraintSystem {
/// A map of variables (IR values) to their corresponding index in the
/// constraint system.
- DenseMap<Value *, unsigned> Value2Index;
+ SmallDenseMap<Value *, unsigned, 16> Value2Index;
// Eliminate constraints from the system using Fourier–Motzkin elimination.
bool eliminateUsingFM();
@@ -70,7 +70,7 @@ class ConstraintSystem {
Value2Index.insert({Arg, Value2Index.size() + 1});
}
}
- ConstraintSystem(const DenseMap<Value *, unsigned> &Value2Index)
+ ConstraintSystem(const SmallDenseMap<Value *, unsigned, 16> &Value2Index)
: NumVariables(Value2Index.size()), Value2Index(Value2Index) {}
bool addVariableRow(ArrayRef<int64_t> R) {
@@ -92,8 +92,8 @@ class ConstraintSystem {
return true;
}
- DenseMap<Value *, unsigned> &getValue2Index() { return Value2Index; }
- const DenseMap<Value *, unsigned> &getValue2Index() const {
+ SmallDenseMap<Value *, unsigned, 16> &getValue2Index() { return Value2Index; }
+ const SmallDenseMap<Value *, unsigned, 16> &getValue2Index() const {
return Value2Index;
}
diff --git a/llvm/include/llvm/Analysis/DominanceFrontier.h b/llvm/include/llvm/Analysis/DominanceFrontier.h
index 68ddcf753b59f7..394379d9750827 100644
--- a/llvm/include/llvm/Analysis/DominanceFrontier.h
+++ b/llvm/include/llvm/Analysis/DominanceFrontier.h
@@ -41,8 +41,8 @@ class DominanceFrontierBase {
public:
// Dom set for a bb. Use SetVector to make iterating dom frontiers of a bb
// deterministic.
- using DomSetType = SetVector<BlockT *>;
- using DomSetMapType = DenseMap<BlockT *, DomSetType>; // Dom set map
+ using DomSetType = SmallSetVector<BlockT *, 16>;
+ using DomSetMapType = SmallDenseMap<BlockT *, DomSetType, 16>; // Dom set map
protected:
using BlockTraits = GraphTraits<BlockT *>;
diff --git a/llvm/include/llvm/Analysis/DominanceFrontierImpl.h b/llvm/include/llvm/Analysis/DominanceFrontierImpl.h
index e877b2c4749abe..165e137a5ecfc0 100644
--- a/llvm/include/llvm/Analysis/DominanceFrontierImpl.h
+++ b/llvm/include/llvm/Analysis/DominanceFrontierImpl.h
@@ -55,7 +55,7 @@ void DominanceFrontierBase<BlockT, IsPostDom>::print(raw_ostream &OS) const {
OS << " <<exit node>>";
OS << " is:\t";
- const SetVector<BlockT *> &BBs = I->second;
+ const SmallSetVector<BlockT *, 16> &BBs = I->second;
for (const BlockT *BB : BBs) {
OS << ' ';
diff --git a/llvm/include/llvm/Analysis/LoopIterator.h b/llvm/include/llvm/Analysis/LoopIterator.h
index 523d2a21825d0d..aaca0298d21da4 100644
--- a/llvm/include/llvm/Analysis/LoopIterator.h
+++ b/llvm/include/llvm/Analysis/LoopIterator.h
@@ -97,8 +97,8 @@ struct LoopBodyTraits {
class LoopBlocksDFS {
public:
/// Postorder list iterators.
- typedef std::vector<BasicBlock*>::const_iterator POIterator;
- typedef std::vector<BasicBlock*>::const_reverse_iterator RPOIterator;
+ typedef SmallVector<BasicBlock*, 16>::const_iterator POIterator;
+ typedef SmallVector<BasicBlock*, 16>::const_reverse_iterator RPOIterator;
friend class LoopBlocksTraversal;
@@ -108,8 +108,8 @@ class LoopBlocksDFS {
/// Map each block to its postorder number. A block is only mapped after it is
/// preorder visited by DFS. It's postorder number is initially zero and set
/// to nonzero after it is finished by postorder traversal.
- DenseMap<BasicBlock*, unsigned> PostNumbers;
- std::vector<BasicBlock*> PostBlocks;
+ SmallDenseMap<BasicBlock*, unsigned, 16> PostNumbers;
+ SmallVector<BasicBlock*, 16> PostBlocks;
public:
LoopBlocksDFS(Loop *Container) :
diff --git a/llvm/include/llvm/Analysis/MemorySSA.h b/llvm/include/llvm/Analysis/MemorySSA.h
index 09fc34af60dc3c..1ceabfa1b13cf9 100644
--- a/llvm/include/llvm/Analysis/MemorySSA.h
+++ b/llvm/include/llvm/Analysis/MemorySSA.h
@@ -879,7 +879,7 @@ class MemorySSA {
Loop *L = nullptr;
// Memory SSA mappings
- DenseMap<const Value *, MemoryAccess *> ValueToMemoryAccess;
+ SmallDenseMap<const Value *, MemoryAccess *, 16> ValueToMemoryAccess;
// These two mappings contain the main block to access/def mappings for
// MemorySSA. The list contained in PerBlockAccesses really owns all the
@@ -895,7 +895,7 @@ class MemorySSA {
// Note that the numbering is local to a block, even though the map is
// global.
mutable SmallPtrSet<const BasicBlock *, 16> BlockNumberingValid;
- mutable DenseMap<const MemoryAccess *, unsigned long> BlockNumbering;
+ mutable SmallDenseMap<const MemoryAccess *, unsigned long, 16> BlockNumbering;
// Memory SSA building info
std::unique_ptr<ClobberWalkerBase> WalkerBase;
diff --git a/llvm/include/llvm/Analysis/MustExecute.h b/llvm/include/llvm/Analysis/MustExecute.h
index 8ac3c5eb653cd4..e49b6facf76b53 100644
--- a/llvm/include/llvm/Analysis/MustExecute.h
+++ b/llvm/include/llvm/Analysis/MustExecute.h
@@ -58,7 +58,7 @@ class raw_ostream;
/// methods except for computeLoopSafetyInfo is undefined.
class LoopSafetyInfo {
// Used to update funclet bundle operands.
- DenseMap<BasicBlock *, ColorVector> BlockColors;
+ BlockColorMapT BlockColors;
protected:
/// Computes block colors.
@@ -66,7 +66,7 @@ class LoopSafetyInfo {
public:
/// Returns block colors map that is used to update funclet operand bundles.
- const DenseMap<BasicBlock *, ColorVector> &getBlockColors() const;
+ const BlockColorMapT &getBlockColors() const;
/// Copy colors of block \p Old into the block \p New.
void copyColors(BasicBlock *New, BasicBlock *Old);
diff --git a/llvm/include/llvm/IR/EHPersonalities.h b/llvm/include/llvm/IR/EHPersonalities.h
index c70f832de40b40..2dcb7511ba4dfc 100644
--- a/llvm/include/llvm/IR/EHPersonalities.h
+++ b/llvm/include/llvm/IR/EHPersonalities.h
@@ -107,12 +107,13 @@ inline bool isNoOpWithoutInvoke(EHPersonality Pers) {
bool canSimplifyInvokeNoUnwind(const Function *F);
typedef TinyPtrVector<BasicBlock *> ColorVector;
+typedef SmallDenseMap<BasicBlock *, ColorVector, 16> BlockColorMapT;
/// If an EH funclet personality is in use (see isFuncletEHPersonality),
/// this will recompute which blocks are in which funclet. It is possible that
/// some blocks are in multiple funclets. Consider this analysis to be
/// expensive.
-DenseMap<BasicBlock *, ColorVector> colorEHFunclets(Function &F);
+BlockColorMapT colorEHFunclets(Function &F);
} // end namespace llvm
diff --git a/llvm/include/llvm/IR/PredIteratorCache.h b/llvm/include/llvm/IR/PredIteratorCache.h
index ba3228347076b8..055ab91a522668 100644
--- a/llvm/include/llvm/IR/PredIteratorCache.h
+++ b/llvm/include/llvm/IR/PredIteratorCache.h
@@ -26,7 +26,7 @@ namespace llvm {
/// wants the predecessor list for the same blocks.
class PredIteratorCache {
/// Cached list of predecessors, allocated in Memory.
- DenseMap<BasicBlock *, ArrayRef<BasicBlock *>> BlockToPredsMap;
+ SmallDenseMap<BasicBlock *, ArrayRef<BasicBlock *>, 16> BlockToPredsMap;
/// Memory - This is the space that holds cached preds.
BumpPtrAllocator Memory;
diff --git a/llvm/include/llvm/Transforms/Utils/Cloning.h b/llvm/include/llvm/Transforms/Utils/Cloning.h
index a4be24e32c5279..3f27cc0cf449db 100644
--- a/llvm/include/llvm/Transforms/Utils/Cloning.h
+++ b/llvm/include/llvm/Transforms/Utils/Cloning.h
@@ -81,7 +81,7 @@ struct ClonedCodeInfo {
/// Like VMap, but maps only unsimplified instructions. Values in the map
/// may be dangling, it is only intended to be used via isSimplified(), to
/// check whether the main VMap mapping involves simplification or not.
- DenseMap<const Value *, const Value *> OrigVMap;
+ SmallDenseMap<const Value *, const Value *, 16> OrigVMap;
ClonedCodeInfo() = default;
diff --git a/llvm/include/llvm/Transforms/Utils/SSAUpdaterImpl.h b/llvm/include/llvm/Transforms/Utils/SSAUpdaterImpl.h
index 746926e5bee331..089a7d05ca0949 100644
--- a/llvm/include/llvm/Transforms/Utils/SSAUpdaterImpl.h
+++ b/llvm/include/llvm/Transforms/Utils/SSAUpdaterImpl.h
@@ -70,7 +70,7 @@ class SSAUpdaterImpl {
: BB(ThisBB), AvailableVal(V), DefBB(V ? this : nullptr) {}
};
- using AvailableValsTy = DenseMap<BlkT *, ValT>;
+ using AvailableValsTy = SmallDenseMap<BlkT *, ValT, 16>;
AvailableValsTy *AvailableVals;
diff --git a/llvm/lib/Analysis/InlineCost.cpp b/llvm/lib/Analysis/InlineCost.cpp
index d2c329ba748e58..3ad137382cd5fc 100644
--- a/llvm/lib/Analysis/InlineCost.cpp
+++ b/llvm/lib/Analysis/InlineCost.cpp
@@ -608,7 +608,7 @@ class InlineCostCallAnalyzer final : public CallAnalyzer {
/// The mapping of caller Alloca values to their accumulated cost savings. If
/// we have to disable SROA for one of the allocas, this tells us how much
/// cost must be added.
- DenseMap<AllocaInst *, int> SROAArgCosts;
+ SmallDenseMap<AllocaInst *, int, 16> SROAArgCosts;
/// Return true if \p Call is a cold callsite.
bool isColdCallSite(CallBase &Call, BlockFrequencyInfo *CallerBFI);
diff --git a/llvm/lib/Analysis/LazyValueInfo.cpp b/llvm/lib/Analysis/LazyValueInfo.cpp
index 30dc4ae30dbfa5..d7e663d60d8dc8 100644
--- a/llvm/lib/Analysis/LazyValueInfo.cpp
+++ b/llvm/lib/Analysis/LazyValueInfo.cpp
@@ -322,7 +322,7 @@ class LazyValueInfoImpl {
SmallVector<std::pair<BasicBlock*, Value*>, 8> BlockValueStack;
/// Keeps track of which block-value pairs are in BlockValueStack.
- DenseSet<std::pair<BasicBlock*, Value*> > BlockValueSet;
+ SmallDenseSet<std::pair<BasicBlock*, Value*>, 16> BlockValueSet;
/// Push BV onto BlockValueStack unless it's already in there.
/// Returns true on success.
diff --git a/llvm/lib/Analysis/MustExecute.cpp b/llvm/lib/Analysis/MustExecute.cpp
index caed62679a683c..52d631b56beb7b 100644
--- a/llvm/lib/Analysis/MustExecute.cpp
+++ b/llvm/lib/Analysis/MustExecute.cpp
@@ -28,7 +28,7 @@ using namespace llvm;
#define DEBUG_TYPE "must-execute"
-const DenseMap<BasicBlock *, ColorVector> &
+const BlockColorMapT &
LoopSafetyInfo::getBlockColors() const {
return BlockColors;
}
diff --git a/llvm/lib/CodeGen/LiveDebugValues/InstrRefBasedImpl.cpp b/llvm/lib/CodeGen/LiveDebugValues/InstrRefBasedImpl.cpp
index a9d28a39c4418b..2b65d2ed45b516 100644
--- a/llvm/lib/CodeGen/LiveDebugValues/InstrRefBasedImpl.cpp
+++ b/llvm/lib/CodeGen/LiveDebugValues/InstrRefBasedImpl.cpp
@@ -4151,7 +4151,7 @@ std::optional<ValueIDNum> InstrRefBasedLDV::resolveDbgPHIsImpl(
// Adapted LLVM SSA Updater:
LDVSSAUpdater Updater(Loc, MLiveIns);
// Map of which Def or PHI is the current value in each block.
- DenseMap<LDVSSABlock *, BlockValueNum> AvailableValues;
+ SmallDenseMap<LDVSSABlock *, BlockValueNum, 16> AvailableValues;
// Set of PHIs that we have created along the way.
SmallVector<LDVSSAPhi *, 8> CreatedPHIs;
diff --git a/llvm/lib/CodeGen/MachineSSAUpdater.cpp b/llvm/lib/CodeGen/MachineSSAUpdater.cpp
index 4cbb6ad3128bd9..186f729859092e 100644
--- a/llvm/lib/CodeGen/MachineSSAUpdater.cpp
+++ b/llvm/lib/CodeGen/MachineSSAUpdater.cpp
@@ -34,7 +34,7 @@ using namespace llvm;
#define DEBUG_TYPE "machine-ssaupdater"
-using AvailableValsTy = DenseMap<MachineBasicBlock *, Register>;
+using AvailableValsTy = SmallDenseMap<MachineBasicBlock *, Register, 16>;
static AvailableValsTy &getAvailableVals(void *AV) {
return *static_cast<AvailableValsTy*>(AV);
diff --git a/llvm/lib/CodeGen/WinEHPrepare.cpp b/llvm/lib/CodeGen/WinEHPrepare.cpp
index c58c67b70fe3c2..07057d4fe69375 100644
--- a/llvm/lib/CodeGen/WinEHPrepare.cpp
+++ b/llvm/lib/CodeGen/WinEHPrepare.cpp
@@ -90,7 +90,7 @@ class WinEHPrepareImpl {
EHPersonality Personality = EHPersonality::Unknown;
const DataLayout *DL = nullptr;
- DenseMap<BasicBlock *, ColorVector> BlockColors;
+ BlockColorMapT BlockColors;
MapVector<BasicBlock *, std::vector<BasicBlock *>> FuncletBlocks;
};
@@ -189,7 +189,7 @@ static BasicBlock *getCleanupRetUnwindDest(const CleanupPadInst *CleanupPad) {
static void calculateStateNumbersForInvokes(const Function *Fn,
WinEHFuncInfo &FuncInfo) {
auto *F = const_cast<Function *>(Fn);
- DenseMap<BasicBlock *, ColorVector> BlockColors = colorEHFunclets(*F);
+ BlockColorMapT BlockColors = colorEHFunclets(*F);
for (BasicBlock &BB : *F) {
auto *II = dyn_cast<InvokeInst>(BB.getTerminator());
if (!II)
diff --git a/llvm/lib/IR/EHPersonalities.cpp b/llvm/lib/IR/EHPersonalities.cpp
index 7c32601b8a83ea..08cbd92cc38baf 100644
--- a/llvm/lib/IR/EHPersonalities.cpp
+++ b/llvm/lib/IR/EHPersonalities.cpp
@@ -102,10 +102,10 @@ bool llvm::canSimplifyInvokeNoUnwind(const Function *F) {
return !EHa && !isAsynchronousEHPersonality(Personality);
}
-DenseMap<BasicBlock *, ColorVector> llvm::colorEHFunclets(Function &F) {
+BlockColorMapT llvm::colorEHFunclets(Function &F) {
SmallVector<std::pair<BasicBlock *, BasicBlock *>, 16> Worklist;
BasicBlock *EntryBlock = &F.getEntryBlock();
- DenseMap<BasicBlock *, ColorVector> BlockColors;
+ BlockColorMapT BlockColors;
// Build up the color map, which maps each block to its set of 'colors'.
// For any block B the "colors" of B are the set of funclets F (possibly
diff --git a/llvm/lib/IR/Verifier.cpp b/llvm/lib/IR/Verifier.cpp
index b89c9ce46e7d61..2d594945ca839f 100644
--- a/llvm/lib/IR/Verifier.cpp
+++ b/llvm/lib/IR/Verifier.cpp
@@ -363,7 +363,7 @@ class Verifier : public InstVisitor<Verifier>, VerifierSupport {
/// Cache which blocks are in which funclet, if an EH funclet personality is
/// in use. Otherwise empty.
- DenseMap<BasicBlock *, ColorVector> BlockEHFuncletColors;
+ BlockColorMapT BlockEHFuncletColors;
/// Cache of constants visited in search of ConstantExprs.
SmallPtrSet<const Constant *, 32> ConstantExprVisited;
diff --git a/llvm/lib/Target/X86/X86WinEHState.cpp b/llvm/lib/Target/X86/X86WinEHState.cpp
index 963d613ddbfe7d..199f8438c61e8e 100644
--- a/llvm/lib/Target/X86/X86WinEHState.cpp
+++ b/llvm/lib/Target/X86/X86WinEHState.cpp
@@ -70,9 +70,9 @@ class WinEHStatePass : public FunctionPass {
bool isStateStoreNeeded(EHPersonality Personality, CallBase &Call);
void rewriteSetJmpCall(IRBuilder<> &Builder, Function &F, CallBase &Call,
Value *State);
- int getBaseStateForBB(DenseMap<BasicBlock *, ColorVector> &BlockColors,
+ int getBaseStateForBB(BlockColorMapT &BlockColors,
WinEHFuncInfo &FuncInfo, BasicBlock *BB);
- int getStateForCall(DenseMap<BasicBlock *, ColorVector> &BlockColors,
+ int getStateForCall(BlockColorMapT &BlockColors,
WinEHFuncInfo &FuncInfo, CallBase &Call);
// Module-level type getters.
@@ -501,7 +501,7 @@ void WinEHStatePass::rewriteSetJmpCall(IRBuilder<> &Builder, Function &F,
// Figure out what state we should assign calls in this block.
int WinEHStatePass::getBaseStateForBB(
- DenseMap<BasicBlock *, ColorVector> &BlockColors, WinEHFuncInfo &FuncInfo,
+ BlockColorMapT &BlockColors, WinEHFuncInfo &FuncInfo,
BasicBlock *BB) {
int BaseState = ParentBaseState;
auto &BBColors = BlockColors[BB];
@@ -520,7 +520,7 @@ int WinEHStatePass::getBaseStateForBB(
// Calculate the state a call-site is in.
int WinEHStatePass::getStateForCall(
- DenseMap<BasicBlock *, ColorVector> &BlockColors, WinEHFuncInfo &FuncInfo,
+ BlockColorMapT &BlockColors, WinEHFuncInfo &FuncInfo,
CallBase &Call) {
if (auto *II = dyn_cast<InvokeInst>(&Call)) {
// Look up the state number of the EH pad this unwinds to.
@@ -644,7 +644,7 @@ void WinEHStatePass::addStateStores(Function &F, WinEHFuncInfo &FuncInfo) {
calculateWinCXXEHStateNumbers(&F, FuncInfo);
// Iterate all the instructions and emit state number stores.
- DenseMap<BasicBlock *, ColorVector> BlockColors = colorEHFunclets(F);
+ BlockColorMapT BlockColors = colorEHFunclets(F);
ReversePostOrderTraversal<Function *> RPOT(&F);
// InitialStates yields the state of the first call-site for a BasicBlock.
diff --git a/llvm/lib/Transforms/InstCombine/InstructionCombining.cpp b/llvm/lib/Transforms/InstCombine/InstructionCombining.cpp
index 1f4a6f793404cf..fbd11292115c2b 100644
--- a/llvm/lib/Transforms/InstCombine/InstructionCombining.cpp
+++ b/llvm/lib/Transforms/InstCombine/InstructionCombining.cpp
@@ -5307,7 +5307,7 @@ bool InstCombinerImpl::prepareWorklist(Function &F) {
bool MadeIRChange = false;
SmallPtrSet<BasicBlock *, 32> LiveBlocks;
SmallVector<Instruction *, 128> InstrsForInstructionWorklist;
- DenseMap<Constant *, Constant *> FoldedConstants;
+ SmallDenseMap<Constant *, Constant *, 16> FoldedConstants;
AliasScopeTracker SeenAliasScopes;
auto HandleOnlyLiveSuccessor = [&](BasicBlock *BB, BasicBlock *LiveSucc) {
diff --git a/llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp b/llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp
index 2ad89b5ba753a5..4665304632c9bd 100644
--- a/llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp
+++ b/llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp
@@ -669,7 +669,7 @@ class RuntimeCallInserter {
return;
assert(TrackInsertedCalls && "Calls were wrongly tracked");
- DenseMap<BasicBlock *, ColorVector> BlockColors = colorEHFunclets(*OwnerFn);
+ BlockColorMapT BlockColors = colorEHFunclets(*OwnerFn);
for (CallInst *CI : InsertedCalls) {
BasicBlock *BB = CI->getParent();
assert(BB && "Instruction doesn't belong to a BasicBlock");
diff --git a/llvm/lib/Transforms/Instrumentation/PGOInstrumentation.cpp b/llvm/lib/Transforms/Instrumentation/PGOInstrumentation.cpp
index 10442fa0bb9003..164ffbec26421a 100644
--- a/llvm/lib/Transforms/Instrumentation/PGOInstrumentation.cpp
+++ b/llvm/lib/Transforms/Instrumentation/PGOInstrumentation.cpp
@@ -864,7 +864,7 @@ BasicBlock *FuncPGOInstrumentation<Edge, BBInfo>::getInstrBB(Edge *E) {
// value profiling call for the value profile candidate call.
static void
populateEHOperandBundle(VPCandidateInfo &Cand,
- DenseMap<BasicBlock *, ColorVector> &BlockColors,
+ BlockColorMapT &BlockColors,
SmallVectorImpl<OperandBundleDef> &OpBundles) {
auto *OrigCall = dyn_cast<CallBase>(Cand.AnnotatedInst);
if (!OrigCall)
@@ -1006,7 +1006,7 @@ void FunctionInstrumenter::instrument() {
// Windows exception handling attached to them. However, if value profiling is
// inserted for one of these calls, then a funclet value will need to be set
// on the instrumenta...
[truncated]
|
You can test this locally with the following command:git-clang-format --diff 774893dcd929c370bad714a70a7d670bb2d6f649 306e47077161c6ce4f727b7551bfe34c3747c4db --extensions cpp,h -- clang/include/clang/AST/CXXInheritance.h clang/lib/AST/ItaniumMangle.cpp llvm/include/llvm/ADT/SCCIterator.h llvm/include/llvm/Analysis/ConstraintSystem.h llvm/include/llvm/Analysis/DominanceFrontier.h llvm/include/llvm/Analysis/DominanceFrontierImpl.h llvm/include/llvm/Analysis/LoopIterator.h llvm/include/llvm/Analysis/MemorySSA.h llvm/include/llvm/Analysis/MustExecute.h llvm/include/llvm/IR/EHPersonalities.h llvm/include/llvm/IR/PredIteratorCache.h llvm/include/llvm/Transforms/Utils/Cloning.h llvm/include/llvm/Transforms/Utils/SSAUpdaterImpl.h llvm/lib/Analysis/InlineCost.cpp llvm/lib/Analysis/LazyValueInfo.cpp llvm/lib/Analysis/MustExecute.cpp llvm/lib/CodeGen/LiveDebugValues/InstrRefBasedImpl.cpp llvm/lib/CodeGen/MachineSSAUpdater.cpp llvm/lib/CodeGen/WinEHPrepare.cpp llvm/lib/IR/EHPersonalities.cpp llvm/lib/IR/Verifier.cpp llvm/lib/Target/X86/X86WinEHState.cpp llvm/lib/Transforms/InstCombine/InstructionCombining.cpp llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp llvm/lib/Transforms/Instrumentation/PGOInstrumentation.cpp llvm/lib/Transforms/ObjCARC/ObjCARC.cpp llvm/lib/Transforms/ObjCARC/ObjCARC.h llvm/lib/Transforms/ObjCARC/ObjCARCContract.cpp llvm/lib/Transforms/ObjCARC/ObjCARCOpts.cpp llvm/lib/Transforms/Scalar/ConstraintElimination.cpp llvm/lib/Transforms/Scalar/Reassociate.cpp llvm/lib/Transforms/Utils/SSAUpdater.cppView the diff from clang-format here.diff --git a/llvm/include/llvm/Analysis/LoopIterator.h b/llvm/include/llvm/Analysis/LoopIterator.h
index aaca0298d2..b641cd5231 100644
--- a/llvm/include/llvm/Analysis/LoopIterator.h
+++ b/llvm/include/llvm/Analysis/LoopIterator.h
@@ -97,8 +97,8 @@ struct LoopBodyTraits {
class LoopBlocksDFS {
public:
/// Postorder list iterators.
- typedef SmallVector<BasicBlock*, 16>::const_iterator POIterator;
- typedef SmallVector<BasicBlock*, 16>::const_reverse_iterator RPOIterator;
+ typedef SmallVector<BasicBlock *, 16>::const_iterator POIterator;
+ typedef SmallVector<BasicBlock *, 16>::const_reverse_iterator RPOIterator;
friend class LoopBlocksTraversal;
@@ -108,8 +108,8 @@ private:
/// Map each block to its postorder number. A block is only mapped after it is
/// preorder visited by DFS. It's postorder number is initially zero and set
/// to nonzero after it is finished by postorder traversal.
- SmallDenseMap<BasicBlock*, unsigned, 16> PostNumbers;
- SmallVector<BasicBlock*, 16> PostBlocks;
+ SmallDenseMap<BasicBlock *, unsigned, 16> PostNumbers;
+ SmallVector<BasicBlock *, 16> PostBlocks;
public:
LoopBlocksDFS(Loop *Container) :
diff --git a/llvm/lib/Analysis/LazyValueInfo.cpp b/llvm/lib/Analysis/LazyValueInfo.cpp
index d7e663d60d..81899e9fe1 100644
--- a/llvm/lib/Analysis/LazyValueInfo.cpp
+++ b/llvm/lib/Analysis/LazyValueInfo.cpp
@@ -322,7 +322,7 @@ class LazyValueInfoImpl {
SmallVector<std::pair<BasicBlock*, Value*>, 8> BlockValueStack;
/// Keeps track of which block-value pairs are in BlockValueStack.
- SmallDenseSet<std::pair<BasicBlock*, Value*>, 16> BlockValueSet;
+ SmallDenseSet<std::pair<BasicBlock *, Value *>, 16> BlockValueSet;
/// Push BV onto BlockValueStack unless it's already in there.
/// Returns true on success.
diff --git a/llvm/lib/Analysis/MustExecute.cpp b/llvm/lib/Analysis/MustExecute.cpp
index 52d631b56b..d08165633c 100644
--- a/llvm/lib/Analysis/MustExecute.cpp
+++ b/llvm/lib/Analysis/MustExecute.cpp
@@ -28,8 +28,7 @@ using namespace llvm;
#define DEBUG_TYPE "must-execute"
-const BlockColorMapT &
-LoopSafetyInfo::getBlockColors() const {
+const BlockColorMapT &LoopSafetyInfo::getBlockColors() const {
return BlockColors;
}
diff --git a/llvm/lib/Target/X86/X86WinEHState.cpp b/llvm/lib/Target/X86/X86WinEHState.cpp
index 199f8438c6..2181d7a70d 100644
--- a/llvm/lib/Target/X86/X86WinEHState.cpp
+++ b/llvm/lib/Target/X86/X86WinEHState.cpp
@@ -70,10 +70,10 @@ private:
bool isStateStoreNeeded(EHPersonality Personality, CallBase &Call);
void rewriteSetJmpCall(IRBuilder<> &Builder, Function &F, CallBase &Call,
Value *State);
- int getBaseStateForBB(BlockColorMapT &BlockColors,
- WinEHFuncInfo &FuncInfo, BasicBlock *BB);
- int getStateForCall(BlockColorMapT &BlockColors,
- WinEHFuncInfo &FuncInfo, CallBase &Call);
+ int getBaseStateForBB(BlockColorMapT &BlockColors, WinEHFuncInfo &FuncInfo,
+ BasicBlock *BB);
+ int getStateForCall(BlockColorMapT &BlockColors, WinEHFuncInfo &FuncInfo,
+ CallBase &Call);
// Module-level type getters.
Type *getEHLinkRegistrationType();
@@ -500,9 +500,8 @@ void WinEHStatePass::rewriteSetJmpCall(IRBuilder<> &Builder, Function &F,
}
// Figure out what state we should assign calls in this block.
-int WinEHStatePass::getBaseStateForBB(
- BlockColorMapT &BlockColors, WinEHFuncInfo &FuncInfo,
- BasicBlock *BB) {
+int WinEHStatePass::getBaseStateForBB(BlockColorMapT &BlockColors,
+ WinEHFuncInfo &FuncInfo, BasicBlock *BB) {
int BaseState = ParentBaseState;
auto &BBColors = BlockColors[BB];
@@ -519,9 +518,8 @@ int WinEHStatePass::getBaseStateForBB(
}
// Calculate the state a call-site is in.
-int WinEHStatePass::getStateForCall(
- BlockColorMapT &BlockColors, WinEHFuncInfo &FuncInfo,
- CallBase &Call) {
+int WinEHStatePass::getStateForCall(BlockColorMapT &BlockColors,
+ WinEHFuncInfo &FuncInfo, CallBase &Call) {
if (auto *II = dyn_cast<InvokeInst>(&Call)) {
// Look up the state number of the EH pad this unwinds to.
assert(FuncInfo.InvokeStateMap.count(II) && "invoke has no state!");
diff --git a/llvm/lib/Transforms/Instrumentation/PGOInstrumentation.cpp b/llvm/lib/Transforms/Instrumentation/PGOInstrumentation.cpp
index 164ffbec26..8ca2663bb5 100644
--- a/llvm/lib/Transforms/Instrumentation/PGOInstrumentation.cpp
+++ b/llvm/lib/Transforms/Instrumentation/PGOInstrumentation.cpp
@@ -863,8 +863,7 @@ BasicBlock *FuncPGOInstrumentation<Edge, BBInfo>::getInstrBB(Edge *E) {
// funclet information, if any is needed, that should be placed on the generated
// value profiling call for the value profile candidate call.
static void
-populateEHOperandBundle(VPCandidateInfo &Cand,
- BlockColorMapT &BlockColors,
+populateEHOperandBundle(VPCandidateInfo &Cand, BlockColorMapT &BlockColors,
SmallVectorImpl<OperandBundleDef> &OpBundles) {
auto *OrigCall = dyn_cast<CallBase>(Cand.AnnotatedInst);
if (!OrigCall)
diff --git a/llvm/lib/Transforms/ObjCARC/ObjCARC.cpp b/llvm/lib/Transforms/ObjCARC/ObjCARC.cpp
index 4745bb9bbc..86149e820f 100644
--- a/llvm/lib/Transforms/ObjCARC/ObjCARC.cpp
+++ b/llvm/lib/Transforms/ObjCARC/ObjCARC.cpp
@@ -21,10 +21,11 @@
using namespace llvm;
using namespace llvm::objcarc;
-CallInst *objcarc::createCallInstWithColors(
- FunctionCallee Func, ArrayRef<Value *> Args, const Twine &NameStr,
- BasicBlock::iterator InsertBefore,
- const BlockColorMapT &BlockColors) {
+CallInst *objcarc::createCallInstWithColors(FunctionCallee Func,
+ ArrayRef<Value *> Args,
+ const Twine &NameStr,
+ BasicBlock::iterator InsertBefore,
+ const BlockColorMapT &BlockColors) {
FunctionType *FTy = Func.getFunctionType();
Value *Callee = Func.getCallee();
SmallVector<OperandBundleDef, 1> OpBundles;
diff --git a/llvm/lib/Transforms/ObjCARC/ObjCARC.h b/llvm/lib/Transforms/ObjCARC/ObjCARC.h
index fe08aec6d4..ebddff65cc 100644
--- a/llvm/lib/Transforms/ObjCARC/ObjCARC.h
+++ b/llvm/lib/Transforms/ObjCARC/ObjCARC.h
@@ -97,10 +97,10 @@ static inline MDString *getRVInstMarker(Module &M) {
/// Create a call instruction with the correct funclet token. This should be
/// called instead of calling CallInst::Create directly unless the call is
/// going to be removed from the IR before WinEHPrepare.
-CallInst *createCallInstWithColors(
- FunctionCallee Func, ArrayRef<Value *> Args, const Twine &NameStr,
- BasicBlock::iterator InsertBefore,
- const BlockColorMapT &BlockColors);
+CallInst *createCallInstWithColors(FunctionCallee Func, ArrayRef<Value *> Args,
+ const Twine &NameStr,
+ BasicBlock::iterator InsertBefore,
+ const BlockColorMapT &BlockColors);
class BundledRetainClaimRVs {
public:
@@ -117,9 +117,9 @@ public:
CallBase *AnnotatedCall);
/// Insert a retainRV/claimRV call with colors.
- CallInst *insertRVCallWithColors(
- BasicBlock::iterator InsertPt, CallBase *AnnotatedCall,
- const BlockColorMapT &BlockColors);
+ CallInst *insertRVCallWithColors(BasicBlock::iterator InsertPt,
+ CallBase *AnnotatedCall,
+ const BlockColorMapT &BlockColors);
/// See if an instruction is a bundled retainRV/claimRV call.
bool contains(const Instruction *I) const {
diff --git a/llvm/lib/Transforms/ObjCARC/ObjCARCContract.cpp b/llvm/lib/Transforms/ObjCARC/ObjCARCContract.cpp
index 60ba0a8752..513361b41b 100644
--- a/llvm/lib/Transforms/ObjCARC/ObjCARCContract.cpp
+++ b/llvm/lib/Transforms/ObjCARC/ObjCARCContract.cpp
@@ -85,19 +85,18 @@ class ObjCARCContract {
SmallPtrSet<CallInst *, 8> StoreStrongCalls;
/// Returns true if we eliminated Inst.
- bool tryToPeepholeInstruction(
- Function &F, Instruction *Inst, inst_iterator &Iter,
- bool &TailOkForStoreStrong,
- const BlockColorMapT &BlockColors);
+ bool tryToPeepholeInstruction(Function &F, Instruction *Inst,
+ inst_iterator &Iter, bool &TailOkForStoreStrong,
+ const BlockColorMapT &BlockColors);
bool optimizeRetainCall(Function &F, Instruction *Retain);
bool contractAutorelease(Function &F, Instruction *Autorelease,
ARCInstKind Class);
- void tryToContractReleaseIntoStoreStrong(
- Instruction *Release, inst_iterator &Iter,
- const BlockColorMapT &BlockColors);
+ void tryToContractReleaseIntoStoreStrong(Instruction *Release,
+ inst_iterator &Iter,
+ const BlockColorMapT &BlockColors);
public:
bool init(Module &M);
@@ -414,8 +413,7 @@ void ObjCARCContract::tryToContractReleaseIntoStoreStrong(
bool ObjCARCContract::tryToPeepholeInstruction(
Function &F, Instruction *Inst, inst_iterator &Iter,
- bool &TailOkForStoreStrongs,
- const BlockColorMapT &BlockColors) {
+ bool &TailOkForStoreStrongs, const BlockColorMapT &BlockColors) {
// Only these library routines return their argument. In particular,
// objc_retainBlock does not necessarily return its argument.
ARCInstKind Class = GetBasicARCInstKind(Inst);
diff --git a/llvm/lib/Transforms/Scalar/ConstraintElimination.cpp b/llvm/lib/Transforms/Scalar/ConstraintElimination.cpp
index 452750ba5a..71e397f873 100644
--- a/llvm/lib/Transforms/Scalar/ConstraintElimination.cpp
+++ b/llvm/lib/Transforms/Scalar/ConstraintElimination.cpp
@@ -288,7 +288,8 @@ public:
SmallDenseMap<Value *, unsigned, 16> &getValue2Index(bool Signed) {
return Signed ? SignedCS.getValue2Index() : UnsignedCS.getValue2Index();
}
- const SmallDenseMap<Value *, unsigned, 16> &getValue2Index(bool Signed) const {
+ const SmallDenseMap<Value *, unsigned, 16> &
+ getValue2Index(bool Signed) const {
return Signed ? SignedCS.getValue2Index() : UnsignedCS.getValue2Index();
}
@@ -892,8 +893,9 @@ void ConstraintInfo::transferToOtherSystem(
#ifndef NDEBUG
-static void dumpConstraint(ArrayRef<int64_t> C,
- const SmallDenseMap<Value *, unsigned, 16> &Value2Index) {
+static void
+dumpConstraint(ArrayRef<int64_t> C,
+ const SmallDenseMap<Value *, unsigned, 16> &Value2Index) {
ConstraintSystem CS(Value2Index);
CS.addVariableRowFill(C);
CS.dump();
|
I'm not going to fix all the clang-format errors as there's a lot of surrounding cruft that I haven't actually changed.
kuhar
reviewed
Oct 10, 2024
Member
kuhar
left a comment
kuhar
left a comment
There was a problem hiding this comment.
Could you open a separate PR the code formatting changes? This will make it easier to review.
This reverts commit f484375.
Member
Author
|
Undid the format-changes in this PR. For context: I've been building profiles of all DenseMap allocations across a build of the CTMark suite to find DenseMaps that a) are frequently used and b) usually have a small number of elements, making them good candidates for initial stack allocations. This patch one of the results of that profile-building, the touched DenseMaps are all reasonably frequently used, and using SmallDenseMap yields a small compile-time improvement. |
nikic
reviewed
Oct 16, 2024
| class OverridingMethods { | ||
| using ValuesT = SmallVector<UniqueVirtualMethod, 4>; | ||
| using MapType = llvm::MapVector<unsigned, ValuesT>; | ||
| using MapType = llvm::SmallMapVector<unsigned, ValuesT, 16>; |
Contributor
There was a problem hiding this comment.
It looks like this is nested inside another MapVector. I'd drop this one if it doesn't have significant effect.
| /// A map of variables (IR values) to their corresponding index in the | ||
| /// constraint system. | ||
| DenseMap<Value *, unsigned> Value2Index; | ||
| SmallDenseMap<Value *, unsigned, 16> Value2Index; |
Contributor
There was a problem hiding this comment.
I'd add a using/typedef for this one, especially as it needs to be synchronized with ConstraintElimination.
Member
Author
|
/me squints -- this was a minor improvement and I've lost all the context behind it, abandoning as it's not worth revisiting IMO |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
These DenseMaps all appear as some of the most frequent sources of memory-allocations that could otherwise be accomodated with an initial stack allocation. For simplicity, I've typedef'd one map-type to be BlockColorMapT, used by various block colouring functions.
This also features one opportunistic replacement of std::vector with SmallVector, as it's in a path that obviously always allocates.
Compared to the other patches reducing DenseMap allocations the benefits from this one are quite small -- I've reached the point of diminishing returns: https://llvm-compile-time-tracker.com/compare.php?from=c7fbcae72557cbe14bca615038254649c1177401&to=0785a8d0fd31541d6a8af616111312780e268611&stat=instructions:u